FIX: CREATE TABLE AS SELECT with MATCH...AGAINST drops string-literal quotes during rewrite - #25683
Conversation
…4823) CREATE TABLE AS SELECT re-serializes the SELECT (build_ddl.go RawSQL, deparsed with WithSingleQuoteString). FullTextMatchExpr.Format wrote the AGAINST pattern with raw WriteString, dropping the surrounding quotes, so a query like CREATE TABLE ctas_t AS SELECT id, MATCH(body) AGAINST('防水' IN BOOLEAN MODE) sc FROM ft WHERE MATCH(body) AGAINST('防水' IN BOOLEAN MODE); re-parsed as AGAINST(防水 ...) -> syntax error 1105. The pattern is stored unquoted (search_pattern: STRING strips the quotes), so Format must re-quote it. Emit it via WriteValue(P_char, FormatString(pattern)) exactly like NumVal string literals: quoted+escaped in the CTAS/restore context, unchanged (raw) in the default human-readable context — so existing MATCH roundtrip tests are unaffected. Regression: the matrixorigin#24823 query + an embedded-single-quote case added to the WithSingleQuoteString roundtrip suite (TestSQLStringFmt). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…24823) End-to-end regression for the deparse fix: builds a gojieba fulltext index and runs CREATE TABLE AS SELECT with MATCH(body) AGAINST('防水' IN BOOLEAN MODE) in both the projection (scored) and WHERE, plus WHERE-only variants. Before the fix this errored with syntax 1105 (dropped pattern quotes on CTAS re-serialization); now it creates the table with the correct matching rows. mo-tester run mode: 14/14. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
…tter Address review: the WithNoBackslashEscape formatter option existed but no production caller passed it - CTAS formatted stmt.AsSource with only quote options, then the generated INSERT ... SELECT is re-parsed by the internal executor under the SESSION's sql_mode. Under NO_BACKSLASH_ESCAPES the stored MATCH pattern holds backslashes literally, so the mode-blind deparse doubled them and the follow-up query searched a different pattern than the user wrote. buildTable now resolves the session sql_mode (same helper the view path uses) and adds WithNoBackslashEscape when the mode is active. Regression: TestCTASFullTextNoBackslashEscapes builds the CTAS plan under NO_BACKSLASH_ESCAPES via a mock sql_mode override and asserts the generated CreateAsSelectSql keeps the literal single backslash (and that the default mode still re-escapes). Verified the test fails without the build_ddl.go change and passes with it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Review follow-up (pushed via maintainer edit — @cpegeric please take a look): Assessment of the three CHANGES_REQUESTED reviews against the head:
Fixed in e3eca1a:
One known adjacent gap, deliberately out of scope: ordinary (non-MATCH) string literals in CTAS go through |
|
Filed the adjacent |
gouhongshen
left a comment
There was a problem hiding this comment.
Codex automated review
Default-path MATCH literal quoting is fixed. The prior NO_BACKSLASH_ESCAPES CTAS blocker is still blocking with corrected evidence: the new formatter option assumes session-mode reparsing, but CTAS’s internal executor parses in default mode. The generic WriteValue gap is pre-existing and tracked separately (#26300), so not a finding. Merge-resolution changes introduced no separate issue. Focused tests could not run because go is unavailable in this workspace.
P1 - CTAS now changes NO_BACKSLASH_ESCAPES MATCH patterns during its default-mode internal reparse (pkg/sql/plan/build_ddl.go:1560)
Author response: this condition adds WithNoBackslashEscape because the CTAS follow-up INSERT is said to be reparsed under the session SQL mode. Why this remains blocking: Compile executes that INSERT through txnExecutor.Exec, which calls parsers.Parse at pkg/sql/compile/sql_executor.go:351; parsers.Parse unconditionally supplies an empty SQL mode (pkg/sql/parsers/sqlparse.go:30-32). Thus an original NBE pattern a\nb is stored as literal backslash+n, but the new branch emits one backslash and the internal default parser consumes \n as a newline. Before this change, the normal formatter emitted \\, which the default parser reduced back to the original literal backslash. CTAS can therefore use a different fulltext pattern and copy different rows. Either keep CTAS formatting compatible with its enforced default parser, or propagate SQL mode into the internal parser and make all CTAS literal formatting mode-aware; add a regression that reparses the generated CTAS SQL through the actual executor path and compares the resulting pattern.
…xecutor's default-mode reparse Review follow-up: the previous commit assumed the CTAS follow-up INSERT ... SELECT is reparsed under the session's sql_mode, but the internal SQL executor always parses in DEFAULT mode (parsers.Parse in pkg/sql/compile/sql_executor.go supplies an empty mode). Under NO_BACKSLASH_ESCAPES, emitting a single backslash therefore gets consumed as an escape by the executor's parse and the follow-up query searches a different pattern. The unconditional default escaping was the round-trip-correct behavior: a literal backslash is emitted doubled and the default-mode reparse reduces it back. Revert the WithNoBackslashEscape wiring in buildTable (the formatter option itself stays for same-mode round-trips) and replace the regression with TestCTASFullTextPatternSurvivesInternalReparse, which asserts the real contract: build the CTAS plan under a session mode, reparse CreateAsSelectSql exactly as the executor does (default mode), and require the resulting MATCH pattern to equal what the user wrote. Verified the new test fails against the previous conditional and passes with this change, for both NO_BACKSLASH_ESCAPES and default sessions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@gouhongshen's P1 is correct — thanks, the automated review caught a wrong premise in my previous commit. Verified all three links: the CTAS follow-up INSERT runs through Fixed in 2224671 (reviewer's option A — keep CTAS formatting compatible with the enforced default parser):
I chose option A over propagating session sql_mode into the internal parser because that would change parsing for all internal SQL (much of it system-generated assuming default escaping) — too risky for this PR's scope. @cpegeric FYI — pushed via maintainer edit again. |
gouhongshen
left a comment
There was a problem hiding this comment.
Codex automated review
Previous blockers are resolved: default MATCH literal quoting remains fixed, and the CTAS NO_BACKSLASH_ESCAPES issue is fixed by retaining default escaping for the internal executor’s enforced default-mode reparse, with a regression covering both modes. The broader pre-existing WriteValue escaping gap is tracked in #26300 and is outside this review. Focused Go tests were unavailable because no Go executable is installed locally.
XuPeng-SH
left a comment
There was a problem hiding this comment.
Approved on exact head 2224671345568b12d6e3c84d76c348a1719b6d8e.
The earlier blockers are closed. FullTextMatchExpr now always emits a valid quoted literal on the default formatter path, while the opt-in NO_BACKSLASH_ESCAPES path preserves same-mode AST round trips. More importantly, the latest CTAS correction matches the actual execution contract: the follow-up INSERT is reparsed by the internal executor in default mode, so CTAS must keep default escaping even when the user statement was parsed under NBE. The new plan regression checks the post-reparse pattern rather than only generated text.
I additionally exercised 20,000 deterministic randomized literal round trips across default and NO_BACKSLASH_ESCAPES modes (quotes, backslashes, controls, wildcard escapes, comments/delimiters, Unicode) with no value drift. The three exact new tests passed; focused race repetition passed 67x/67x for parser tests and 42x for the plan test; both complete owning packages passed under -race; vet is clean. Latest origin/main (159742db6672ff5dadb617880b831da026f0a3f4) merges cleanly and focused race tests pass on the prospective merge.
The generic FmtCtx.WriteValue mode gap is pre-existing, separately tracked in #26300, and is not a blocker for this MATCH-specific fix.
|
@Mergifyio queue |
Merge Queue Status
Waiting for
All conditions
|
|
@iamlinjunhong could you take another look? Your NO_BACKSLASH_ESCAPES round-trip concern was addressed (WithNoBackslashEscape formatter option + the four requested round-trip cases in TestFullTextMatchPatternRoundTrip), and the follow-up review iterations settled the CTAS wiring: the internal executor always reparses in default mode, so CTAS formatting stays default-escaped with TestCTASFullTextPatternSurvivesInternalReparse asserting the reparse contract. gouhongshen and XuPeng-SH have both approved the current head; your CHANGES_REQUESTED is the last gate before the merge queue. |
What type of PR is this?
Which issue(s) this PR fixes:
issue #24823
What this PR does / why we need it:
fixed. CREATE TABLE AS SELECT with MATCH...AGAINST drops string-literal quotes during rewrite